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INTRODUCTION 


This communication is the final report for NASA Ames Research Center 
Grant NSG->2118 to Massachusetts Institute of Technology, the period 
covered by the grant extending from November, 1975 to November, 1980. The 
research was all carried out by graduate students in the MIT Man-Machine 
Systems Laboratory. It resulted in two Ph.D. theses and four master's 
theses. All the research was in collaboration with and supervised by the 
principal xnvestigator. Indirectly the various research projects also 
contributed as laboratory demonstrations in a graduate course on 
man-machine system modeling. 

The generic theme of the research was how automation in the cockpit 
is changing the task of the pilot and affecting his performance. In 
general we may say that the pilot's task is changing from that of a 
continuous in-the-loop controller to that of a "flight manager" or 
supervisor of automatic systems. The natural evolution of technology has 
provided him more and different systems, mostly electronic, to program, to 
monitor, to interpret the results from and to be concerned about the 
failure of. Further, within any one system, the pilot may have more 
alternative modes or ways to control certain variables - in some cases, 
e.g. flight control, whole hierarcies of commands and displays ranging 
from pure and direct manual control to inertial navigation and autoland. 
Presumably all these new avionic aids are to make the flight more reliable 
and more efficient, though the charge has been made sometimes that they 
are "killing the pilot with kindness." 

Along with new automation there has been an effort to devise 
experimental paradigms and conceptual and mathematical models which may 
help generalize research results and predict performance under various 
hypothetical system designs. That is where this research claims to have 
made its greatest contributions. There are various aspects of these 
models which may be mentioned here and which reappear in the several 
studies to be cited specifically. One aspect is the "multi-task" quality 


of modorn flight fflanagament, i.e., that pilots are faced with numerous 
semi- independent tasks some of which must be done in parallel and some in 
.sequence, and the attentional demands of which are partly predictable (can 
be planned in part) and partly unpredictable. The successful pilot "keeps 
ahead of the airplane" by anticipating what tasks can be planned and 
attended to ahead of time, leaving sufficient margin for those tasks 
demanding attention at unpredictable times or at a pace which is forced by 
circumstance and which the pilot cannot set. 

Another recurrent theme is that of "mental workload", an ill defined, 
difficult- to-measure, inferred or hypothetical or "intervening" property 
of pilot behavior which nevertheless is of increasing interest. The 
apparent reason is that as we automate the pilot out of routine skill 
tasks we seem to keep adding more cognitive and monitoring tasks which may 
at times be dull and undemanding but at other times can be demanding and 
stressful to the breakpoint. 

Six separate studies are summarized below. Each was published 
earlier as an individual technical report. 

1. IS THERE AN OPTIMAL WORK- LOAD IN MANUAL CONTROL? (Verplank, 1977) 

Most people would agree that somewhere, between boring and fatiguing, 
there is an optimum mental work-load for manual control tasks such as 
driving a car, flying an airplane or regulating a processing plant. In 
particular, automation raises this question for the designer of 
man-machine systems: Is it possible for tasks to be too simple? 

Currently used concepts and measures of work-load are inappropriate to 
predicting, for example, the effects of underload. 

It was proposed at the outset of this study that there is an aspect 
of work-load that sustains performance and facilltiates transitions to new 
tasks or emergencies. Information transmission rate was proposed as the 
apprvopriato measure for this kind of "facilitating v/ork-load". It was 
shown how the information .ransmitted can come not only from the external 
disturbances but also from the human operator’s unr:ertainty in his own 


output. Measures were developed for both external and Internal sources. 


Two experiments with simulated driving tasks and varying wounts of 
external disturbance were used to study long-term performance and 
transitions. Results showed that average performance over the run was 
always best when there was no Input disturbance. The rate of 
deterioration In performance over the run could not be distinguished on 
the basis of the disturbance conditions. In fact, the results suggested 
that tasks of Intermediate difficulty may produce worse decrements than 
either easier or more difficult tasks (just the opposite to what the 
above-stated hypothesis of an optimum would predict). The relative 
variability of performance was one performance measure where the no- input 
condition produced worse performance, yet the difficult condition was 
best, so no intermediate optimum disturbance was found. Performance upon 
transition was confused by many factors which made it difficult to say 
that the differences were due to work-load. For example, with no warning 
the intermediate pre-conditons may be best, with warning worst. Also, it 
inay be that work-load is simply without meaning in explaining the details 
of transition performance. 

The application of the information measures developed did not help 
much in sorting out or explaining the results, the only correlation found 
was between the slope of performance for the no-input case, and the amount 
of internal information transmitted. Greater information transmission 
correlated with greater performance decrement, just opposite to what the 
faciliation hypothesis would predict. 

Several new directions for human factors research were suggested. 
Better models are needed for what is meant by work-load, for how to 
measure it and for what aspects of performance can be predicted. 
Information transmission may not be of value in explaining the temporal 
details of performance, but may help in quantifying activation or 
facilitation aspects of operator loading, internal sources of uncertainty 
help to explain "self-loading" and important limitations in other areas of 
human operator performance such as monitoring, learning and adaptive 
behavior. 


2. DYNAMIC DECISION MAXING IN MULTI*TASK SUPERVISORY CONTROL: COMPARISON 

OF AN OPTIMAL ALGORITHM TO HUMAN BEHAVIOR (Tulga, 1978; Tulga and 

Sheridan, 1980) 

In this research a multi-task dynamic decision paradigm was developed 
to represent the systems within which pilot and other decision makers 
would seem to operate* This paradigm was described in graph-theoretical 
context. It was shown that, like some other combinatiorial optimization 
problems including Job-Shop Scheduling, Traveling Salesman, etc., it too 
is representable as a multi-path decision process. We called the 
algorithm "T-Path". 

Since one of the properties of the T-Paths is the dynamic nature of 
the graph describing them, was shown that estimation of future tasK.s and 
discount rate on returns from planned sequences of actio is are necessary. 

A dynamic programming optimization algorithm was develjped which takes 
into consideration both the currently available and estimated tasks and 
gives an optimal path to follow, under the given model parameters and*)^, 
the latter being the urgency factor for tasks closer to the deadline. 

Using a computer-graphic display for the experimental system, 
experiments were done with human subjects under different task- loads to 
identify the par^uneters and i at which model task choices best matched 
the human data. In addition to control theoretical identification 
criteria, two new criteria were introduced to account for the multi-task 
nature of the problem: (i) percentage of time operator attended the same 

tasks at the same instants and (ii) percentage of tasks that are attended 
eventually by both the model and the human operator. Experimental 
results were found to be fit easily by the "T-Path" model. 

To differentiate several cases studied, an objective 'loading factor* 
was developed, which reduced to the utilization factor of Queueing Theory 
as a special case. Also, a new Work-Load Hypothesis was introduced to 
describe and predict subjective human opinion of mental work load as a 
function of this loading factor. One remarable result from these 


experiments was that, while subjective work load increased with objective 
loading factors so lorg as the operator could "keep up", as loading factor 
Increased further the subjective rating of work load declined, as though 
the operator progressively concerned himself less and less as his relative 
performance deteriorated. 


3. MAN-MACHINE PERFORMANCE FOR A SIMULATED AIRCRAFT WITH MULTI-LE'/EL 
AUTOrATIC CONTROL SYSTEM (Yoerger, 1979) 


This work examined the supervisory control hierarchy of flight 
control, i.e., the ever more prevalent situation where a large number of 
control configurations are simultaneously available to tne pilot. Which 
mode should he use, when? 

A digital simulation of a YC-15 transport aircraft with control wheel 
steering was implemented, with flight instruments and map displays shown 
on a vector graphics terminal. Additionally, three more automatic control 
configurations were also simulated. This allowed for a systematic 
Increase in automatic control of the same dynamic system. 

An experiment was conducted to determine the effects of automatic 
control mode, wind disturbance level, and inter-subject differences on 
both performance and subjective assessment of task demand. Subjects were 
required to follow a three-dimensional path. Performance was found to 
improve and task demand was found to decrease when the subjects were given 
direct set-point control over state variables which also corresponded to 
the task such as heading and vertical speed, as opposed to control inputs 
which related less directly to the task, such as vehicle attitude. 

Transitions from higher to lower levels of automation were 
experimentally examined. It was theorized that differences in accuracy of 
the human's internal representation of the system state for different 
levels of automation would be revealed by observing transitions to the 
least automatic mode after failures had occurred in several more automated 
modes. No differences were found, however. By observing transitions from 


the most automated mode to different, less automated modes, the 
suitability of these less automated inodes for failure recovery could be 
.tested. 

Two control configurations were found to be unsuitable for failure 
recovery. Pitch angle setpoint control resulted in poor performance, 
primarily due to display/task incompatibility. Heading setpoint control 
was unsuicable due to the inability of the subjects to quickly choose a 
new heading from the information displayed and enter it with the available 
control device. 

4. SUBJECTIVE SCALING OF MENTAL WORKLOAD IN A MULTI-TASK ENVIRONMENT 

(Daryanian, 1980) 

In this research we sought to determine the differential weightings 
of subjective mental workload factors which entered into the "objective 
workload" index used by Tulga in his abstract multi-task experiments. 

These were; (1) average interarrival time between appearance of new blocks 
(tasks); (2) speed with which blocks moved toward the deadline; and (3) 
productivity, or rate at which blocks diminished to zero when attended to 
(cursor was set on them). All 27 combinations of three levels of each 
factor were used, each combination being paired with each other 
combination for all possible pairs. Three subjects, after 100 seconds at 
each of two such tasks, used a five-level scale to rate which had the 
highest workload, and to rate the relative certainty, 1 being certainly 
of one stimulus, 5 being certainly for the other, 3 being indifference 
between stimuli. 

A classical Tnurstonlan ("law of compartive judgement") scaling 
procedure was used. The reader is referred to the referenced report for a 
primer on this scaling technique. Essentially a Thurstone scale orders the 
judged stimuli while using consistency in ordering any two vaules to 
weight the psychological distance between those two values along the 
'Judgement scale. The sum of such weightings determines the location of 
any stimulus on the judgement scale. 


Exparlmental results revealed rather strikingly that one factor, the 
Interarrlval tine, correlated quJte highly with mental workload as derived 
by the Thurstonian scaling transformations. Speed and productivity did 
not so correlate. This was evidently due to the fact that strategies 
changed radically with interarrival time, but did not change so radically 
for the other factors, at least with the ranges of each employed in Ui« 
experiments. Response transitivity was "east (subjectr had most 
difficulty in deciding which stimulus of a pair had greatest mental 
workload) when interarrival time was short (and speed and productivity 
varied) . 

5. CONTROL MODELS OF AIRCRAFT IN-TRAIL FOLLOWING (Buharali, 1980) 

This report applied the theory of optimal control to modelling a 
string of airplanes following one another in-trail. 

The work was motivated by interest in providing to commercial pilots 
computer-based displays of nearby traffic, plus maps and time-lines, by 
means of which he presumably can perform better longitudinal control 
relative to separations and schedules. Unfortunately, while there was 
communication with personnel at several NASA aviation planning groups, no 
empirical in-trail following data were available, either from actual or 
simulated flight, to fit the models. It is hoped that some 
to-be- conducted simulations incorporating the cockpit display of traffic 
information (CDTI) will provide such data, and that the models will find 
direct usefulness thereby. 

Current models of in-trail following are based only on the 
longitudional separation from a single leading aircraft. The work here 
described is novel in that it considers control criteria (objective 
functions) which include for each aircraft in a string: (1) distance and 
velocity relative to the immediately leading aircraft; (2) distance and 
velocity .’elative to the aircraft in front of that one; (3) deviation from 
reference position and velocity; and (4) control effort. 

Three different versions of the model made different assumptions 



about what the "lead pilot" in a kitring will do relative to responding to 
disturbances, and whether he will return to steady-state reference 
position and velocity. 

Further variations in the modelling effort include: (1) use ol' a 
threshold on position and velocity error signals below which a pilot does 
not control, and (2) use of dual control law (a different criterion for 
separation closure than for separation opening) . 

Factors determining practical parameter values are discussed in the 
full report and some parameters are derived. Based on these par2uneters a 
number of computer simulation runs are presented graphically for the 
various models described above. 

6. USE OF A DISAGGREGATED MODEL AS AN AID TO THE HUMAN OPERATOR IN 
FAILURE DIAGNOSIS IN COMPLEX AUTOMATED SYSTEMS (Hasan, 1981) 

Supervisory control requires sufficient attention to the detection 
and diagnosis at faulty system components before they can cause 
irreparable damage. 

This work studied a *disaggregated-model' - based diagnostic 
technique to help the human operator in detection and isolation of faulty 
system components in aircraft and other large scale systems. A 
disaggregated model, in contrast to a complete model, corresponds to 
separation of a physical system model into disparate process. Such models 
can be simulated continuously in parallel or at the discretion of the 
human operator. Inputs to these models are the measured variables at 
corresponding locations throughout the system. Output from these models 
are then dynamically compared against corresponding variables from the 
physical system. The human operator plays the dual role of system 
monitor and failure monitor in the implementation of the proposed 
technique. 

An experimental demonstration was conducted using a simulated 
aircraft to investigate the feasibility and usefulness of the proposed 


technlq<ia. specifically, the effect, of aide based on different levels of 
disaggregation^ on the operator perfortaance in failure diagnosis was 
determined, along with three other factors of interest, namely the level 
of external disturbance and sensor noise, the failure size, and whether 
the operator served as monitor or in- the- loop controller. 

The dynamics of a YC-15 transport aircraft were simulated with 
control-wheel steering as well as a setpoint autopilot control. The 
fll^t instruments, course map and other variables of Interest were 
displayed dynamically on a color-raster graphics terminal. Diagnostic aids 
based on three different levels of disaggregation were implemented. 
Parameters, controlled by potentiometer settings, were made to fail in 
random appearing fashion as trained subjects flew the simulated aircraft 
along predefined paths. 

Operator performance was evaluated in terms of the response times for 
failure detection and isolation and the number of correct failed-parameter 
isolations. Aid based on disaggregated models was found to provide 
significantly improved performance when compared to aid based on a 
complete model. The isolation time was found to be more sensitive to the 
level of disaggregation than the detection time. Effect of failure size 
(magnitude of discrepancy) was found to be insignificant. Noise level was 
moderately significant in failure detection but was insignificant in 
failure isoltion. 
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